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ABSTRACT 

Individual  operations  in  NASS  surveys  may  be  contacted  on  numerous  occasions  over  time.  This 
may  be  particularly  true  with  large  or  unique  operations  possibly  being  selected  with  near 
certainty  for  recurring  surveys  and  included  in  samples  for  multiple  surveys.  Cooperation  in  any 
particular  survey  may  be  affected  by  the  number  and  frequency  of  times  an  establishment  has  been 
selected  for  NASS  surveys  in  the  past. 

This  paper  examines  the  relationship  between  response  on  the  1 998  June  Crops/Stocks  Survey  in 
South  Dakota  and  the  reporting  burden  placed  on  operations  by  NASS  in  the  past.  This  paper 
will  examine  the  effects  that  the  accumulated  reporting  burden  associated  with  the  number  and 
frequency  of  NASS  survey  contacts,  length  of  those  contacts  and  the  difficulty  of  the  survey 
request  have  on  survey  cooperation.  Comparisons  of  these  burden  measures  made  between 
respondents  and  non-respondents  for  the  1998  June  Crops/Stocks  Survey  show  little,  if  any, 
correlation  between  these  burden  measures  and  an  individual  operation’s  willingness  to  cooperate. 
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This  report  is  based  on  a  paper  entitled.  An  Analysis  of  the  Relationship  Between  Survey  Burden 
and  Non-Response:  If  We  Bother  Them  More,  Are  They  Less  Cooperative?  which  was  prepared 
for  the  1999  International  Conference  on  Survey  Non-Response  held  in  Portland,  Oregon, 
October  28-31,  1999. 


SUMMARY 


A  commonly  held  belief  in  survey  research  is  that  increased  burden  is  negatively  correlated 
with  survey  cooperation.  In  this  context,  burden  includes  such  things  as  the  number  and 
frequency  of  survey  contacts,  length  of  those  contacts  and  the  difficulty  of  the  survey  request. 
However,  existing  literature  provides  limited  evidence  to  support  the  notion  that  increased  or 
accumulated  burden  also  increases  refusal  rates. 

USDA’s  National  Agricultural  Statistics  Service  (NASS)  contacts  farms  and  ranches  in 
the  United  States  for  many  surveys.  This  paper  explores  the  accumulated  burden  effect  of  6 1 
surveys  conducted  in  South  Dakota  between  January  1 996  and  June  1 998  on  the  refusal  rate  for  a 
particular  survey,  the  June  1998  Crops/Stocks  Survey. 

There  were  14,557  operations  contacted  for  the  61  surveys.  Of  these.  35  percent  were 
contacted  for  only  a  single  survey  while  85  percent  were  contacted  for  five  or  fewer  surveys  in  the 
two  and  one  half  year  period.  Analyses  revealed  the  number  of  surveys  an  operation  was  in  is  a 
poor  predictor  of  the  operation’s  overall  refusal  rate  for  all  NASS  survey  contacts. 

In  looking  at  the  response  to  the  1998  June  Crops/Stocks  Survey,  we  found  that: 

•  there  was  no  statistically  significant  difference  in  the  mean  number  of  surveys  refusals 
and  cooperators  were  in; 

•  there  was  a  marginally  statistically  significant  difference  in  the  mean  number  of  total 
OMB  questionnaire  minutes;  however,  although  the  refusals  had  more  minutes  than 
the  cooperators,  the  magnitude  of  the  difference  was  not  of  practical  significance; 

•  there  was  no  statistically  significant  difference  in  the  length,  measured  in  OMB 
minutes,  of  the  most  recent  survey  for  the  refusals  and  the  cooperators; 

•  there  was  no  statistically  significant  difference  in  the  refusal  rates  on  the  1998  June 
Crops/Stocks  Survey  for  those  operations  who  were  and  who  were  not  in  a  difficult 
survey  during  the  previous  two  and  one  half  year  period; 

•  there  was  a  statistically  significant  difference  in  the  mean  number  days  since  the  most 
recent  contact;  however,  although  the  refusals  had  a  more  recent  contact  than 
cooperators,  the  magnitude  of  the  difference  was  not  of  practical  significance. 

These  results  indicate  that  accumulated  burden  does  not  necessarily  lead  to  higher  refusal 
rates  on  a  particular  survey.  However,  the  authors  caution  generalizing  these  results  to  all  states. 
Instead,  similar  investigations  should  be  carried  out  in  other  states  and  additional  research  be 
conducted  to  determine  other  factors  that  relate  to  refusing  NASS  surveys. 


1.  INTRODUCTION 

A  commonly  held  belief  in  survey 
research  is  that  increased  burden  is 
negatively  correlated  with  survey 
cooperation.  Federal  Statistical  Agencies 
and  others  are  currently  striving  to  lessen  the 
reporting  burden  placed  on  respondents,  as 
per  the  U.S.  Office  of  Management  and 
Budget’s  current  government  wide  goal  of 
five  percent  yearly  reduction  of  information 
collection  burdens.  (Paperwork  Reduction 
Act  of  1995,  see  also  Machin,  1997  for  a 
review  of  survey  burden  reduction  efforts  in 
the  UK.)  Burden  may  be  defined  in  a 
number  of  ways  -  length  of  the  interview  or 
questionnaire,  number  of  contacts,  difficulty 
in  reporting  the  requested  data,  etc.  The 
length  of  the  survey  questionnaire  is  often 
assumed  to  be  positively  correlated  with 
survey  non-response.  However,  the 
evidence  to  support  this  claim  is  inconsistent 
at  best.  A  literature  review  by  Bogen  (1996) 
found  that  while  this  claim  was  supported  in 
some  studies,  other  studies  showed  exactly 
the  opposite.  In  addition,  some  studies 
showed  that  there  was  little  relationship, 
either  positive  or  negative,  between 
questionnaire  length  and  cooperation. 

Survey  burden  may  also  be  defined  as 
the  number  of  survey  contacts.  In  surveys  of 
certain  populations,  sample  units  may  be 
contacted  on  numerous  occasions  over  time. 
This  is  particularly  true  in  surveys  of 
establishments,  where  large  or  unique 
operations  may  be  selected  with  near 
certainty  for  recurring  surveys,  and  may  be 
included  in  samples  for  multiple  surveys. 
Cooperation  in  any  particular  survey  may  be 
affected  by  the  number  and  frequency  of 
times  an  establishment  has  been  selected  for 
surveys  by  that  organization  in  the  past. 

The  sum  of  the  length  of  time  of 


previous  survey  contacts  is  also  assumed  to 
adversely  affect  survey  participation.  The 
most  common  evidence  for  this  comes  from 
panel  survey  response  patterns.  Most  panel 
surveys,  which  contact  respondents  multiple 
times  over  the  course  of  data  collection, 
suffer  from  attrition  from  the  original  sample 
(see  Kalton.  Kasprzyk.  and  McMillen,  1989 
for  a  discussion  of  nonresponse  in  a  variety 
of  panel  surveys).  This  is  taken  to  be 
evidence  that  increased  contacts  result  in 
subsequent  non-response.  Frankel  and  Sharp 
(1981)  also  found  the  length  of  a  single 
completed  survey  interview  was  related  to 
expressed  willingness  to  participate  in  later 
interviews.  Respondents  who  participated  in 
a  25  minute  interview  were  more  likely  to 
agree  to  participate  in  a  future  interview  than 
respondents  participating  in  a  75  minute 
interview.  However,  there  was  little 
difference  in  actual  cooperation  between 
those  who  had  the  long  or  short  initial 
interview  when  later  contacted  for  the 
second  interview. 

There  is  also  some  evidence  that  past 
survey  experiences  may  trigger  future 
refusals.  DeMaio  ( 1 980)  found  that  one  of 
the  most  common  reasons  for  refusing 
interviews  in  later  waves  of  the  CPS  panel, 
was  ‘"unfavorable  past  experiences”  as 
survey  respondents.  However,  although 
respondents  had  obviously  been  in  a  survey 
before  (the  previous  panel  wave), 
“unfavorableness”  may  be  due  to  either  the 
accumulated  burden  of  repeated  interviews 
or  any  one  of  a  plethora  of  other  factors 
(related  to  the  panel  or  other  recalled 
contacts  from  that  organization  or  others). 

Many  panel  surveys  do  not  re-contact 
early  wave  refusals  in  later  waves  of  data 
collection.  However,  for  panel  surveys  that 
do  re-contact  early  wave  refusals,  a 
significant  proportion  of  them  will  respond 
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on  subsequent  contacts  (Presser,  1989). 

This  implies  that  accumulated  burden  does 
not  always  trigger  subsequent  survey 
refusals. 

USDA’s  National  Agricultural 
Statistics  Service  (NASS)  contacts  farms  and 
ranches  in  the  United  States  for  many 
surveys.  For  example,  the  Quarterly 
Agricultural  Survey  (QAS)  collects  data  on 
inventory  and  production;  the  Farm  Labor 
Surv  ey  collects  information  on  hours  worked 
and  wages;  the  Agricultural  Resource 
Management  Study  (ARMS)  collects 
information  on  production  practices, 
chemical  and  pesticide  use,  and  farm 
economics.  A  particular  farm  or  ranch  may 
be  selected  for  any  or  all  of  these  surveys, 
both  within  a  single  year  and  over  multiple 
years.  This  paper  examines  the  relationship 
between  response  on  one  NASS  survey 
(June  1998  QAS)  and  the  reporting  burden 
placed  on  agricultural  operations  by  NASS  in 
the  past. 

In  NASS  surveys,  there  is  some 
evidence  that  accumulated  burden  may 
contribute  to  later  non-response.  During 
1990  and  1991,  the  reasons  given  for 
refusing  to  participate  inNASS’s  ongoing 
Farm  Costs  and  Returns  Survey  (collecting 
detailed  expenditure,  income,  cost  of 
production  and  demographic  data)  were 
collected.  Some  respondents  did  report 
refusing  because  they  had  been  contacted  on 
this  survey  or  other  surveys  in  the  past 
(O’Connor,  1991,  1992).  However,  this  was 
not  the  most  frequently  cited  reason  for 
refusing  to  participate.  (The  number  one 
reason  for  refusing  in  this  hour  long  survey 
was  “too  busy/lack  of  time.”)  Similarly,  in  a 
survey  of  farmers  and  ranchers  in  North  and 
South  Dakota,  self  reported  number  of  past 
USD  A  requests  for  data  was  not  positively 
correlated  with  their  willingness  to  provide 


data  to  USDA  in  the  future  (Jones,  Sheatsley 
and  Stinchcombe,  1979).  This  group  also 
did  not  cite  the  frequency  or  number  of 
survey  requests  as  a  primary  reason  for 
refusing  to  participate  in  USDA  surveys. 
(Invasion  of  privacy  was  the  number  one 
reason  cited  by  this  group  for  refusing  to 
participate.) 

The  number  of  surveys  that  NASS  is 
conducting  in  the  population  of  agricultural 
operators  has  been  increasing,  while  the  size 
of  the  population  continues  to  decrease. 

This  suggests  that  the  number  of  times  an 
operation  may  be  contacted  by  NASS  and 
the  frequency  of  these  contacts  is  increasing 
over  time.  In  order  to  determine  the 
accumulated  burden  on  sampled  operations 
and  how  this  is  related  to  participation  in  a 
subsequent  survey,  this  paper  examines  a  set 
of  contacts  made  by  NASS  with  farm  and 
ranch  operations  in  South  Dakota  from 
January  1996  to  June  1998. 

2.  METHODS 

Interview  disposition  (not  sampled, 
completed,  refused,  non-contact)  was 
recorded  for  each  South  Dakota  agricultural 
operation  for  the  Quarterly  Agricultural 
Surveys,  the  Agricultural  Labor  Surveys,  the 
Hog  Surveys,  the  Cattle  and  Sheep  Surveys, 
the  Agricultural  Resource  Management 
Studies,  and  the  Yield  surveys  conducted  by 
NASS  between  January  1996  and  June 
1998.  Contacts  coded  as  “estimated”  were 
considered  refusals.  (South  Dakota  was 
selected  because  they  have  historically  had  a 
relatively  high  rate  of  survey  refusals.) 

Details  about  these  surveys  appear  in 
Appendix  A.  There  were  61  total  possible 
survey  contacts  in  this  set. 

All  surveys  were  sampled  from  South 
Dakota’s  list  frame  of  farmers  and  ranchers, 
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which  NASS  maintains.  Samples  were 
stratified  based  on  survey-related  control 
data  (usually  size  and  type  of  operation) 
maintained  on  the  list  frame.  Mode  of 
contact  varied,  but  involved  primarily 
telephone  (CATI),  with  limited  face-to-face 
and  mail  for  most  surveys.  The  exception 
was  the  ARMS,  which  was  entirely  face-to- 
face  interviews.  The  sensitivity  and  difficulty 
of  the  surveys  also  varied,  however,  except 
for  the  ARMS,  all  data  collected  are 
generally  readily  available  to  farmers  and 
ranchers.  The  ARMS  collects  extremely 
detailed  income,  expense  and  debt 
information,  in  addition  to  potentially 
sensitive  questions  on  pesticide  and  chemical 
use. 

The  samples  were  not  all 
independent.  For  example,  the  monthly 
agricultural  yield  survey  samples  are  subsets 
of  the  crops/stocks  samples  and  if  operations 
are  in  one  yield  survey,  they  are  in  all  seven 
yield  surveys  in  a  given  crop  year.  This  had 
some  data  collection  implications  since 
interviewers  might  tell  operators  they  would 
likely  be  contacted  again.  However 
interviewers  would  not  always  know  this  fact 
and  this  is  not  covered  under  official  NASS 
policy. 

For  surveys  other  than  the  yield 
surveys,  interviewers  do  not  normally  know 
what  other  surveys  an  operation  may  have 
been  selected  for,  so  they  will  not  be  able  to 
tell  respondents  how  many  contacts  they  may 
have  in  the  future.  The  few  exceptions  to 
this  are  noted  in  the  appendix.  Note:  This 
analysis  does  not  include  ALL  of  the  survey 
contacts  that  NASS  makes  within  this 
population.  However,  these  are  the  surveys 
which  are  part  of  National  estimating 
programs,  with  the  largest  samples  sizes, 
targeting  the  broadest  populations.  Other 
surveys  are  conducted  which  may  target 


specific  specialized  subpopulations,  (e.g. 
large  cattle  feedlots,  horticultural  operations, 
etc.  which  probably  have  little  overlap  with 
the  surveys  we  examined)  or  may  be 
conducted  under  sponsorship  of 
organizations  other  than  NASS.  Most  of 
these  surveys  record  only  completed 
interviews,  so  refusals  and  other  non¬ 
response  cannot  be  identified.  These  limited 
number  of  surveys  are  not  included  here,  but 
may  have  contributed  to  the  accumulated 
burden. 

In  order  to  measure  the  effect  of 
burden  on  later  response,  we  examined 
response  in  the  most  recent  survey  of  our  set 
(June  1998).  Because  we  were  primarily 
interested  in  sampled  operations  that  refused 
to  participate,  not  those  that  we  were  unable 
to  contact,  we  included  only  two  types  of 
response  in  our  analysis:  those  that  were 
contacted  and  provided  survey  data,  and 
those  that  were  contacted  but  refused  to 
participate.  Other  non-contacts  were 
minimal  in  this  survey  and  were  not  included 
in  our  analyses.  The  sample  size  for  the  June 
1998  survey  was  1,460,  with  69.5%  of  the 
records  completed,  26.7%  refused  and  3.8% 
other  non-response. 

Analyses  took  the  sample  design  into 
consideration  and  used  /-tests  with  jackknife 
estimates  for  standard  errors. 

3.  RESULTS 

3.1  Assessment  of  accumulated  NASS 
survey  burden 

Once  we  combined  response 
information  from  the  61  possible  survey 
contacts  we  were  able  to  see  how  many 
times  NASS  had  contacted  potential 
respondents  and  how  they  responded  to 
these  contacts.  The  results  are  shown  in 
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Table  1.  Percent  of  Surveys  Refused  By  Total  Number  of  Survey  Contacts 


N  amber  of 
Survey 
Contacts 

Count 

0 

1 

2 

3 

4 

Number  of  Surv  eys  Refused 

5  6  7  8  9 

10 

11 

12 

13 

1 

5,125 

76.4 

23.6 

2 

2,737 

68.3 

17.9 

13.7 

3 

2,208 

66.7 

14.0 

9.3 

10.0 

4 

1375 

61.7 

15.5 

9.0 

7.1 

6.6 

5 

923 

61.0 

13.4 

5.7 

7.8 

7.3 

4.8 

6 

704 

59.5 

14.3 

7.8 

4.5 

4.3 

5.5 

4.0 

7 

492 

57.3 

11.8 

8.3 

6.1 

4.7 

5.1 

3.3 

3.5 

8 

304 

54.9 

15.5 

8.2 

3.6 

53 

3.3 

3.6 

2.0 

3.6 

9 

230 

53.9 

15.7 

5.2 

5.7 

4.8 

2.6 

3.5 

4.8 

1.7 

23 

10 

161 

48.4 

18.0 

5.6 

3.1 

7.5 

1.9 

1.2 

3.7 

4.3 

3.7 

2.5 

11 

79 

44.3 

16.5 

3.8 

7.6 

7.6 

1.3 

5.1 

3.8 

2.5 

2.5 

2.5 

23 

12 

68 

48.5 

23.5 

2.9 

1.5 

4.4 

2.9 

4.4 

2.9 

1.5 

5.9 

1.5 

13 

59 

47.5 

10.2 

6.8 

3.4 

3.4 

8.5 

1.7 

6.8 

3.4 

5.1 

3.4 

14 

32 

50.0 

9.4 

3.1 

6.3 

3.1 

3.1 

6.3 

3.1 

6.3 

3.1 

3.1 

3.1 

15 

19 

52.6 

5.3 

10.5 

10.5 

53 

10.5 

5.3 

16 

15 

46.7 

13.3 

6.7 

13.3 

13.3 

17 

4 

25.0 

25.0 

25.0 

25.0 

18 

9 

55.6 

11.1 

11.1 

11.1 

11.1 

19 

3 

33.3 

33.3 

33.3 

20 

4 

25.0 

25.0 

25.0 

25.0 

21 

3 

33.3 

66.7 

22 

1 

100.0 

23 

0 

24 

1 

100.0 

25 

1 

1000 

Note  1:  This  analysis  included  61  possible  surveys  from  January  1996  through  June  1998. 

Note  2:  No  respondent  was  in  more  than  25  surveys 

Note  3:  No  respondent  refused  more  than  15  of  the  surveys  they  were  in. 


Table  1 .  The  total  number  of  operations  that 
were  contacted  on  these  surveys  was  14,557. 

The  maximum  number  of  times  a 
respondent  had  been  contacted  was  25, 
although  over  85%  of  respondents  were 
contacted  five  times  or  less  in  the  2  !4  year 
period.  Particularly  striking  in  this  table  is 
the  number  of  operations  that  were 
contacted  only  once.  This  is  by  far  the  most 
common  number  of  times  an  operation  was 
contacted.  It  should  be  noted  also,  that  the 
1997  Census  of  Agriculture  reported  31,284 
farms  in  South  Dakota.  This  indicates  that 
even  though  the  overall  target  population  is 
relatively  small,  there  is  a  sizable  proportion 
of  potential  respondents  who  were  NEVER 
contacted  by  NASS  for  these  surveys  during 
this  time. 

Table  1  also  shows  the  number  of  times 
potential  respondents  refused  to  provide 


survey  data.  The  number  of  people  who 
refused  1 00  percent  of  the  time  they  were 
contacted  (shown  in  bold  on  the  table 
diagonal)  is  very  small.  Surprisingly,  there 
are  no  “100  percent  refusals”  in  the  group  of 
respondents  who  were  contacted  1 2  times  or 
more.  There  also  did  not  appear  to  be  any 
pattern  to  respondents’  willingness  to 
respond.  That  is,  it  did  not  appear  that 
respondents  cooperated  then  started  to 
refuse,  or  vice  versa.  The  cooperate/refuse 
pattern  appears  random  across  the  repeated 
contacts. 

Table  2  shows  the  same  information 
summarized  by  the  percent  of  times  a 
respondent  refused  to  provide  data.  The 
third  column  of  the  table  shows  the 
percentage  of  respondents  who  were  “hard 
core  cooperators”,  providing  survey  data  all 
or  most  of  the  times  they  were  contacted 
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Table  2.  Percent  of  Surveys  Refused  By  Number  of  Survey  Contacts 


Number  of 
Survey 
Contacts 

Count 

0-24% 
(“Hard  Core 
Cooperators”) 

Percent  Refused 
25-49%  50-74% 

75-100% 
(“Hard  Core 
Refusals) 

1 

5,125 

76.4 

23.6 

2 

2.737 

68.3 

17.9 

13.7 

3 

2,208 

66.7 

14.0 

9.3 

10.0 

4 

1,375 

61.7 

15.5 

9.0 

13.7 

5 

923 

74.4 

5.7 

7.8 

12.1 

6 

704 

73.8 

7.8 

8.8 

9.5 

7 

492 

69.1 

14.4 

9.8 

6.8 

8 

304 

70.4 

11.8 

8.6 

9.2 

9 

230 

74.8 

10.5 

6.1 

8.7 

10 

161 

72.0 

10.6 

6.8 

10.5 

11 

79 

64.6 

16.5 

11.4 

7.5 

12 

68 

74.9 

8.8 

8.8 

7.4 

13 

59 

67.9 

13.6 

10.2 

8.5 

14 

32 

68.8 

6.2 

9.4 

15.6 

15 

19 

68.4 

10.5 

5.3 

15.8 

16 

15 

66.7 

26.6 

6.7 

17 

4 

75.0 

25.0 

18 

9 

66.7 

11.1 

22.2 

19 

3 

33.3 

66.6 

20 

4 

25.0 

50.0 

25.0 

21 

3 

100 

22 

1 

100 

23 

0 

24 

1 

100 

25 

1 

100 

(never  refusing,  up  to  refusing  less  than  one 
quarter  of  the  time).  The  far  right  column 
shows  the  percentage  of  operations  who  are 
“hard  core  refusals”  (refusing  3/4  of  the  time 
contacted  or  more).  The  striking  thing  about 
this  table  is  the  consistent  pattern  shown  for 
all  number  of  contacts.  Hard  core 
cooperators  (column  three)  remain  around 
70  percent  regardless  of  the  number  of 
contacts,  just  as  the  hard  core  refusals  are 
fairly  consistent  around  ten  percent. 

Columns  four  and  five  show  lesser  degrees 
of  cooperation  but  are  also  fairly  consistent. 

If,  in  fact,  more  contacts  make  people 
less  cooperative,  then  we  should  see  an 


increase  in  the  percent  of  time  respondents 
refuse  as  the  number  of  contacts  increases. 
We  do  not  see  this  pattern  in  our  data.  (It  is 
interesting  that  the  few  respondents  with  the 
largest  number  of  contacts  in  our  data  set  are 
more  cooperative  than  respondents  with 
fewer  contacts.) 

A  formal  test  of  the  relationship 
between  the  overall  percentage  of  surveys 
refused  and  the  number  of  surveys  an 
operation  was  selected  for  was  performed 
with  a  simple  regression  model.  The  overall 
refusal  percent  was  modeled  by  the  count  of 
the  number  of  surveys  the  operation  was 
selected  for.  The  coefficient  on  the  number 
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of  surveys  was  -0.00542  with  a  poor  R:  of 
0.0015  (p- value  <  0.001).  With  the  large 
number  of  observations  (14,557)  available, 
the  statistical  significance  is  not  surprising. 
This  means  the  number  of  surveys  accounts 
for  less  than  one  percent  of  the  variation  in 
the  refusal  rate.  Since  the  coefficient’s 
magnitude  is  so  small,  the  result  tells  us  very 
little. 

3.2  Effect  of  Burden  on  Survey  Response 

In  order  to  analyze  the  effect  of  prior 
survey  burden  on  a  subsequent  survey 
contact,  we  compared  operations  who 
cooperated  and  provided  survey  data  with 
operations  who  were  contacted,  but  refused 
to  provide  any  data  for  the  June  1998 
Crops/Stocks  survey.  (This  was  the  last 
survey  included  in  our  data  set.)  As  an 
indication  of  accumulated  burden  we  looked 
at  several  things  traditionally  thought  of  as 


burdensome  to  survey  respondents.  The  first 
type  of  burden  was  the  number  of  times  the 
operation  had  been  contacted  by  NASS  in 
the  past  (as  noted  before,  in  our  analysis  this 
included  6 1  possible  survey  contacts  from 
January  1 996  until  this  survey).  There  was 
no  significant  difference  in  the  number  of 
surveys  refusals  or  cooperators  had  been 
contacted  for,  with  the  average  number  just 
under  three  contacts  for  each  group,  as 
shown  in  Table  3.  We  also  compared  the 
total  amount  of  time  they  had  been  asked  to 
spend  responding  in  the  past  (measured  as 
the  total  OMB  approved  estimate  of  minutes 
of  burden  for  each  survey).  This  is  also 
shown  in  Table  3  as  Mean  Number  of 
Minutes.  There  is  a  marginally  significant 
difference  between  refusals  and  cooperators 
in  the  total  accumulated  interview  time,  but 
in  practical  terms,  the  difference  (three  and 
one  half  minutes)  is  negligible. 


Table  3.  Total  Time  Barden  in  the  Two  &  One  Half  Year  Period 


June  1998  Respondents 

Mean  Number 
of  Survevs 

Mean  Number 
of  Minutes 

Cooperators  (n=988) 

2.7260 

50.9015 

Refusals  (n=368) 

2.8807 

54.4394 

Difference  Between 

-0.1548 

-3.5379 

CooDerators  &  Refusals 

(v- value  0.1460) 

(fr-value  0.0910) 

We  also  thought  that  perhaps  the 
burden  imposed  by  the  most  recent  prior 
contacts  might  affect  cooperation.  We 
compared  the  number  of  minutes  (again, 
based  on  official  OMB  estimated  burden)  for 
respondents  and  cooperators  for  the  two 
most  recent  contacts  prior  to  June  1998 
(Table  4). 


The  only  significant  difference 
between  refusals  and  cooperators  was  in  the 
length  of  the  second  most  recent  contact, 
2.2  minutes.  While  statistically  different,  in 
practical  terms  the  difference  is  negligible  as 
it  is  unlikely  that  respondents  would 
distinguish  between  interview  lengths 
differing  by  two  minutes. 
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Table  4.  Length  of  Two  Most  Recent  Snrvey  Contacts 


June  1998  Respondents 

Mean  Minutes. 
Most  Recent 
Contact 

Mean  Minutes. 

2nd  Most  Recent 
Contact 

Cooperators  (n=988) 

10.7137 

6.0871 

Refusals  (n=368) 

10.4355 

8.2872 

Difference 

0.2782 

-2.2002 

(n-value  0.78271 

(n-value  0.01 72) 

Note:  Operations  that  did  not  have  any  other  contacts  in  the  past  214  years 
were  given  0  minutes  for  their  prior  contacts. 


Demaio  (1980)  found  that 
‘"unfavorable”  prior  survey  experiences  were 
related  to  future  survey  cooperation.  One 
way  a  survey  experience  might  be  perceived 
as  unfavorable  to  a  respondent  is  if  they  have 
to  provide  information  that  is  difficult  to 
report,  is  sensitive,  or  requires  a  lot  of  time 
or  effort  to  provide.  While  most  of  the 
surveys  in  our  data  set  have  relatively  short 
estimated  completion  times  and  are  felt  to 
request  information  that  is  relatively  easy  to 
provide,  the  Agricultural  Resource 
Management  Study  (ARMS)  takes  90 
minutes  on  average  to  complete  (with 
interviews  lasting  up  to  several  hours),  and 
asks  for  several  types  of  information  that 
might  be  deemed  ‘sensitive’,  such  as  detailed 
financial  information  and  pesticide  use.  This 
survey  is  widely  believed  in  NASS  to  be 


extremely  burdensome  and  difficult.  For  this 
reason,  we  suspected  that  participation  in  an 
ARMS  survey  might  adversely  affect 
cooperation  on  later  surveys.  To  test  this 
hypothesis,  we  compared  the  refusal  rate  on 
the  June  1998  survey  for  those  operations 
who  had  been  selected  to  report  in  the 
ARMS  in  the  past  year  with  those  who  had 
not.  As  shown  in  Table  5,  operations  who 
had  previously  been  selected  for  the  ARMS 
survey  were  actually  more  likely  to 
participate  in  the  June  1 998  survey  than 
those  who  had  not.  (However,  because  of 
the  NASS  perceived  burden  imposed  by 
being  in  the  ARMS  survey,  the  sample  is 
selected  specifically  to  minimize  the  number 
of  operations  in  both  the  ARMS  and  other 
surveys  such  as  the  June  1998  QAS.) 


Table  5.  Refusal  Rate  By  Selection  Status  in  ARMS  Survey  (for  the  past  year) 


June  1998  Refusal  Rate 

Selected  for  ARMS  (n=  23) 

0.0929 

NOT  Selected  in  ARMS  (n=  1,333) 

0.2265 

Difference 

-0.1336 

(o-value  0.0660) 
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Because  the  number  of  respondents 
in  the  ARMS  surv  ey  in  the  past  year  is  quite 
small.  we  also  compared  the  June  1998 
response  rate  for  operations  who  had  been  in 
the  ARMS  survey  any  time  in  the  past  two 


and  one  half  years.  Response  rates  were 
virtually  identical  for  both  groups,  indicating 
that  prior  inclusion  in  the  ARMS  survey  did 
not  affect  future  survey  participation  (Table 
6). 


Table  6.  Refusal  Rate  By  Selection  Status  in  ARMS  Survey 
(over  the  past  two  and  one  half  years) 


Jljn^  1998  Refusal  Rate 

Selected  in  ARMS  (n=223) 

0.2482 

NOT  Selected  in  ARMS  (n=l,133) 

0.2208 

Difference 

0.0274 

(n-value  0.5391) 

It  has  also  been  suggested  that 
burden  is  increased  with  more  frequent 
contacts  made  to  a  potential  respondent.  In 
order  to  test  this  hypothesis,  we  compared 
the  average  number  of  days  since  the  last 
contact  (and  the  contact  prior  to  that)  had 
been  made  with  June  cooperators  and 
refusals.  As  shown  in  Table  7,  the 


difference  of  45  days  is  statistically 
significant,  but  again,  since  the  average 
length  of  time  since  the  prior  contact  was 
close  to  a  year  and  ten  months,  this  may  have 
little  practical  value  with  regard  to 
developing  contact  procedures  for 
respondents. 


Table  7.  Time  Since  Prior  Contacts 


June  1998  Respondents 

Mean  Days 
Between  Most 
Recent  Contact 

Mean  Days  Between 

2nd  Most  Recent 
Contact 

Cooperators  (n=988) 

681.6458 

772.5490 

Refusals  (n=368) 

636.5099 

753.8431 

Difference  Between 

45.1359 

18.7059 

CooDerators  &  Refusals 

(v- value  0.0238) 

(z?-value  0.1588) 

Note:  Respondents  who  had  not  had  a  prior  contact  in  the  past  2  V2  years  were  given 
a  value  of  913  days  (2  Vi-  years),  since  that  was  the  minimum  time  since  they  would 
have  been  contacted. 
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4.  DISCUSSION 

In  this  paper  we  tried  to  define 
“survey  burden"  in  a  number  of  ways  that  are 
traditionally  used  by  survey  researchers. 

Then  we  examined  these  burdens  (as 
imposed  by  NASS)  to  see  if  they  affected 
cooperation  on  a  future  survey.  If  burden 
imposed  by  NASS  is  correlated  with 
cooperation  on  NASS  surveys,  NASS  may 
want  to  consider  ways  to  decrease  the 
burden  on  individual  operations  as  a  means 
to  increase  later  response.  However,  none 
of  the  types  of  burden  we  looked  at  appeared 
to  be  related  to  future  survey  cooperation  in 
the  1998  June  Crops/Stocks  Survey.  While 
this  is  a  single  survey,  it  is  typical  of  NASS 
surveys  and  is  representative  of  the  type  of 
information  and  respondents  in  most  other 
NASS  surveys. 

Of  course,  our  results  do  not  reflect 
contacts  by  organizations  other  than  NASS; 
we  only  included  contacts  over  which  we 
have  control.  Agricultural  operations  are 
subjected  to  requests  from  other  agricultural 
businesses,  in  addition  to  requests  targeted  at 
the  general  population  We  do  not  know¬ 
how  this  additional  survey  burden  affects 
their  cooperativeness  on  NASS  surveys.  We 
do  know,  from  other  research  (McCarthy, 
Johnson,  and  Ott,  2000),  that  sampled 
operations  who  have  more  knowledge  and 
more  positive  opinions  of  NASS  are  more 
likely  to  cooperate  on  our  surveys. 

In  addition,  there  may  be  other  ways 
in  which  respondent  burden  may  be  defined. 
For  example,  anecdotal  evidence  suggests 
that  the  timing  of  a  survey  contact  may  affect 
its  perceived  burden.  Farmers  may  feel  a 
contact  is  more  burdensome  during 
extremely  busy  times,  such  as  harvesting  or 
planting,  on  the  farm.  Establishments  may 
perceive  respondent  burden  more  in  terms  of 


seasonal  activities  occurring  when  they  are 
contacted  rather  than  by  how  long  or  how 
frequent  the  surveys  are. 

Because  farmers  and  ranchers  are  a 
specialized  population,  they  may  respond 
differently  to  response  burden.  As 
representatives  of  agricultural 
establishments,  they  may  be  less  likely  to 
refuse  cooperation  because  the  survey  topics 
are  personally  relevant  to  them  and  they  may 
realize  ways  in  which  the  survey  results 
affect  them.  This  may  be  true  of  other 
establishment  populations  as  well. 

5.  CONCLUSION  AND 
RECOMMENDATIONS 

Using  the  data  available  to  us,  we 
found  little,  if  any,  evidence  to  support  the 
idea  that  accumulated  burden  adversely 
affects  survey  cooperation.  However,  our 
findings  should  not  be  taken  to  mean  that 
NASS  should  ignore  trying  to  reduce  the 
number  of  times  they  contact  respondents. 
Federal  Statistical  agencies  should  be 
commended  for  efforts  to  reduce  burden  on 
respondents,  since  they  usually  provide  data 
voluntarily  and  without  compensation. 
However,  if  the  objective  of  reducing  burden 
is  to  increase  cooperation,  we  may  be 
disappointed  with  the  results  of  burden 
reduction  efforts. 

One  alternative  strategy  might  be  to 
increase  the  burden  on  a  smaller  group  of 
respondents  and  forgo  the  objective  of 
making  burden  as  small  as  possible  for 
everyone.  If  burden  is  concentrated  on  a 
small  group  within  the  population,  additional 
resources  could  then  be  spent  on  maximizing 
the  cooperation  of  this  smaller  group  who 
would  be  asked  to  participate  in  many  more 
surveys  than  they  currently  are.  Whether  or 
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not  cooperation  could  be  maintained  with 
greatly  increased  amounts  of  data  being 
collected  is  of  course,  unknown.  To  a 
respondent,  it  would  be  similar  to  being 
asked  to  be  in  an  ongoing  panel  survey  (for 
which  high  response  rates  can  be  achieved). 
However,  instead  of  panel  waves,  each 
contact  would  be  for  a  different  and  separate 
survey.  Our  research  suggests  that  this  may 
be  a  fruitful  avenue  for  future  research 
efforts. 

Cooperation  on  agricultural  surveys 
may  be  tied  more  to  other  phenomena  than 
respondent  burden.  We  have  recently  begun 
to  collect  information  about  respondents  and 
non-respondents  attitudes  toward  NASS  as 
the  survey  sponsor  (McCarthy,  Johnson  and 
Ott,  2000).  Differences  in  the  feelings 
potential  respondents  have  about  the  survey 
sponsor  and  the  effect  of  survey  statistics  on 
respondents  appear  to  be  much  more  closely 
related  to  survey  cooperation  or  refusal  than 
burden. 

While  survey  organizations  should 
continue  to  strive  to  reduce  burden  on  their 
respondents,  particularly  those  in  limited 
populations,  we  should  also  be  looking  for 
real  correlates  of  survey  response.  Burden, 
while  long  assumed  to  be  directly  related  to 
survey  cooperation,  may,  in  truth,  have  little 
impact  on  response.  If  that  is  the  case, 
resources  spent  on  burden  reduction  in 
pursuit  of  increased  response  rates  may  be 
money  ill  spent. 

Based  on  our  findings,  we  offer  the 
following  recommendations: 

Recommendation  1 :  Investigate  other  factors 
which  may  be  related  to  refusing  NASS 
surveys. 


Recommendation  2:  Repeat  the  analyses 
discussed  in  this  paper  in  one  or  more  other 
states.  It  is  possible  that  the  results  obtained 
for  South  Dakota  are  unique  and  other  states 
may  follow  different  trends.  Other 
possibilities  for  additional  analysis  include 
examining  response/nonresponse  on  other 
surveys  or  burden  over  other  time  periods. 

Recommendation  3:  Perform  additional 
analyses  on  the  South  Dakota  data  used  for 
this  paper.  Additional  analyses  could  involve 
determining  if  particular  subgroups  react 
differently  to  accumulated  burden. 

Recommendation  4:  Explore  alternatives  to 
minimizing  burden  across  all  respondents. 
Sampling  plans  could  be  designed  so  that 
burden  was  purposefully  increased  on  a  small 
set  of  respondents,  for  which  extra  resources 
would  be  used  to  maintain  their  cooperation. 
(This  approach  is  part  of  the  pilot  for  an 
alternative  to  the  current  ARMS  survey 
design.) 
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